Developing a dancer sonification system using the Immersive Interactive Sonification Platform (iISoP)

For decades, researchers have spurred research on sonification, the use of non-speech audio to convey information. As the level of automation increases in the workplace, monitoring the state and performance of complex systems can require more cognitive resources than provided through visual information processing channels. In recent years interactive sonification and gesture-based interaction have become emerging fields in human-computer interaction. The ultimate goal of the current project is to establish an immersive interactive sonification platform (iISoP) for diverse sonification and gesture controlled design research. Visual representations of the users’ movements are presented on the display wall, while their gestures and affect are translated into sound using a variety of sonification algorithms. The object of this paper is to present and describe the variety of ways movement and affective data can be mapped to sound, and the underlying relationship between sound and movement. I will also discuss the user centered design process that is involved in making an intuitive, expressive, and entertaining dance-based sonification system.